On Speeding up the Em Algorithm in Pattern Recognition: a Comparison of Incremental and Multiresolution Kd-tree-based Approaches

Finite mixture models implemented via the EM algorithm are being increasingly used in a wide range of problems in the context of unsupervised statistical pattern recognition. As each E-step visits each feature vector on a given iteration, the EM algorithm requires considerable computation time in its application to large data sets. We consider two approaches, an incremental EM (IEM) algorithm and a multiresolution kd-tree-based approach, that can be used to reduce the computational time in applying the EM algorithm. In this paper, we investigate and compare their relative performances in speeding up the EM algorithm. Some simulated and real data on medical magnetic resonance (MR) images were used in this investigation. The results show that the IEM algorithm leads to the lowest error rate, but that the reduction in the time to convergence is very limited. The multiresolution kd-tree-based algorithms, on the other hand, provide larger reduction in computation time. In particular, it is demonstrated that by using an IEM algorithm in conjunction with a multiresolution kd-tree, the EM algorithm can be speeded up by a factor of 56.0 for very large data sets.